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MEMORANDUM  FOR  THE  COMPTROLLER,  DEPARTMENT  OF 
DEFENSE 

Via:  Comptroller  of  the  Navy 

Subj:  CINCLANTFLT  FY  96  GOVERNMENT  PERFORMANCE  & 
RESULTS  ACT  (GPRA)  PERFORMANCE  REPORT 

1.  This  report  concludes  Commander  in  Chief,  U.S.  Atlantic  Fleet’s 
(CINCLANTFLT)  formal  participation  in  the  GPRA  FY  96  Performance  Plan 
pilot  series.  It  provides  background  information  on  the  pilot  project 
development  process,  project  highlights,  results  and  lessons  learned  from  the 
participation  of  the  GEORGE  WASHINGTON  Carrier  Battle  Group  as 
CINCLANTFLT’ s  designated  pilot  activity. 

2.  There  are  a  series  of  “Firsts”  in  this  report.  It  marks  Navy’s  initial 
participation  in  the  GPRA  performance  plan  pilot  series  required  by  the  law.  It 
also  highlights  DoD’s  initial  designation  of  a  forward  deployed  combatant  force 
as  a  participant  in  a  GPRA  pilot  program.  Lastly  it  defines  the  process  we  used 
in  our  initial  attempt  to  quantitatively  measure  the  performance  of  a  major  naval 
combatant  force  through  its  operating  cycle  including  overseas  deployment 
under  the  Navy’s  Forward  Presence  Strategy. 

3.  Our  experience  in  this  performance  plan  pilot  event  has  proven  most 
rewarding.  We  have  collectively  learned  much  on  how  to  approach  measuring 
the  performance  of  a  complex,  dynamic  organization  such  as  a  Carrier  Battle 
Group.  This  effort  also  complemented  and  reinvigorated  internal  staff 
initiatives  to  better  measure  the  resource  costs  associated  with  providing  combat 
ready  rotational  forces  for  forward  deployment.  “Cost  visibility”  has  taken  on 
renewed  emphasis  within  the  Atlantic  Fleet  and  subordinate  staffs  as  we  build 
upon  the  knowledge  gained  in  this  GPRA  pilot  project. 
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4.  The  success  gained  from  this  pilot  effort  rests  with  the  GEORGE 
WASHINGTON  Battle  Group  staff.  Their  enthusiastic  acceptance  of  this  added 
challenge  and  their  exceptional  dedication  in  ensuring  the  concept  of  combatant 
force  performance  measurement  was  thoroughly  tested  were  the  key  ingredients 
to  success. 

5.  Additionally  OSD  (C)(PMR)’s  support  in  providing  the  pilot  activities  with 
the  latitude  to  fully  explore  the  opportunities  presented  in  this  pilot  initiative 
was  instrumental  to  our  ability  to  achieve  a  satisfying  end  result. 


HENRY  C.  GIFFIN,  III 
Deputy  and  Chief  of  Staff 
Commander  in  Chief, 

U.S.  Atlantic  Fleet 


Copy  to; 

Chief  of  Naval  Operations  (CNO  NOO,  N09,  N8) 
Commander  in  Chief,  U.S.  Pacific  Fleet  (NOO) 
Commander,  SECOND  FLEET 
Commander,  GEORGE  WASHINGTON  Battle  Group 
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I.  Executive  Summary 

The  GEORGE  WASHINGTON  Carrier  Battle  Group  was  nominated  as 
Commander  in  Chief,  U.S.  Atlantic  Fleet’s  (CINCLANTFLT)  participant  in  the 
GPRA  FY  96  Performance  Plan  pilot  series.  In  response  to  the  Department  of 
Defense’s  initiative  to  include  a  front  line  combatant  force  in  the  pilot  series,  the 
GEORGE  WASHINGTON  Battle  Group  was  selected  based  upon  its  scheduled 
FY  96  training  workup  and  deployment. 

The  challenge  to  quantitatively  measure  the  performance  of  a  complex  and 
operationally  dynamic  organization  like  a  Carrier  Battle  Group  was  a 
formidable  task  given  the  compressed  time  available  and  the  lack  of  an 
institutionalized  assessment  process  to  measure  aggregate  level  performance. 

A  Fleet  Working  Group,  composed  of  warfare  community  subject  matter 
experts  from  the  staffs  of  the  Atlantic  Fleet  Headquarters,  Commander, 
SECOND  FLEET,  and  the  GEORGE  WASHINGTON  Battle  Group,  was 
established  to  address  this  challenge.  The  result  of  their  collective  efforts  was  a 
prototype  performance  assessment  model  including  the  performance  indicators 
and  metrics  designed  to  measure  the  Battle  Group’s  capability  to  successfully 
execute  the  mission  critical  tasks  set  forth  by  the  Chief  of  Naval  Operations  and 
approved  by  the  Joint  Chiefs  of  Staff. 

Performance  goals,  both  overall  and  in  each  selected  critical  task,  were  set  by 
the  Battle  Group  Commander  for  the  duration  of  the  scheduled  six  months 
overseas  deployment.  Formal  data  collection  commenced  on  the  Battle  Group’s 
departure  from  homeport  in  January  1996  and  continued  until  its  return  in  July 
1996. 

All  performance  goals  were  met  and  the  prototype  performance  assessment 
software  program  proved  successful  in  providing  the  Commander  with  an 
improved  management  tool  to  measure  daily  Battle  Group  operations.  Based  on 
the  demonstrated  success  of  the  prototype  system,  we  are  refining  and 
institutionalizing  the  measurement  process  in  all  Atlantic  Fleet  Carrier  Battle 
Groups  and  Amphibious  Ready  Groups. 

From  the  Fleet  Commander  in  Chief  perspective,  the  lessons  learned  from  this 
pilot  have  provided  our  staff  with  invaluable,  practical  experience  in 
performance  measurement  and  a  solid  baseline  from  which  we  can  conceptually 
link  performance  and  resources. 


1 


11.  Project  Background 


In  response  to  DoD’s  initiative  to  include  combatant  force  participation  in  the 
GPRA  FY  96  Performance  Plan  pilot  series,  the  Assistant  Secretary  of  the  Navy 
for  Financial  Management  (ASN  FM)  nominated  Commander  in  Chief,  U.S. 
Atlantic  Fleet  (CINCLANTFLT)  in  October  1994  to  participate  as  Navy’s 
representative  in  the  pilot  project.  To  explore  the  feasibility  of  fully 
implementing  GPRA  within  a  combatant  force,  CINCLANTFLT  nominated  the 
GEORGE  WASHINGTON  Battle  Group  as  the  Navy’s  pilot  activity  based  on 
its  scheduled  FY  96  training  work  up  and  overseas  deployment.  In  January 
1995,  Director,  Office  of  Manpower  and  Budget  (0MB),  approved  the 
nomination  of  the  GEORGE  WASHINGTON  Battle  Group  as  an  FY  96 
participant. 

In  our  initial  status  briefing  to  the  Office  of  the  Secretary  of  Defense  (OSD)  and 
0MB  representatives,  discussion  centered  on  two  factors  related  to  our  ability  to 
fully  engage  in  the  GPRA  performance  plan  pilot  effort. 

First,  GPRA  is  focused  on  comparative  assessment  of  outcome  measures  which 
for  a  combatant  force  would  be  optimally  measured  during  actual  combat 
operations.  Although  prepared  to  conduct  contingency  operations  at  a  moments 
notice.  Navy’s  Forward  Presence  Strategy  is  focused  on  deterring  hostilities.  A 
routine  peacetime  presence  deployment  thus  would  be  expected  to  deny  the 
opportunity  to  measure  the  outcomes  of  combatant  force  operations.  The 
concept  of  measuring  the  “projected  output  capability”  of  the  Battle  Group  to 
execute  its  mission  critical  tasks  as  a  workable  proxy  was  proposed  and 
accepted  by  OSD  and  0MB. 

Second,  Navy  has  not  quantitatively  measured  nor  maintained  historical  data  on 
Battle  Group  operations  on  a  scale  sufficient  to  support  meaningful  comparative 
assessment  of  individual  Battle  Group  performance.  We  also  have  limited 
experience  in  attempting  to  quantify  aggregate  level  performance  within  the 
Battle  Group  Commander’s  domain  of  constantly  changing  variables. 
Participation  in  a  performance  pilot  of  this  scope  and  focus  was  truly  akin  to 
sailing  uncharted  waters. 

CINCLANTFLT’s  Warfare  Programs  &  Readiness  Directorate  (N8)  was  tasked 
to  provide  overall  project  direction,  coordination  and  support  for  pilot  plan 
development  and  implementation.  Commander,  GEORGE  WASHINGTON 
Battle 
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Group  was  assigned  pilot  plan  execution  responsibility  with  CINCLANTFLT 
N8  providing  support  and  assistance  as  necessary.  CINCLANTFLT  support  for 
this 

added  tasking  was  provided  from  inplace  staff  resources,  later  augmented  by  the 
transfer  of  two  civilian  ceiling  points  from  within  the  CINCLANTFLT 
claimancy.  No  added  funding  was  provided  to  support  this  initiative. 
Developmental  work  in  creating  a  performance  assessment  model  and  the 
associated  performance  indicators  and  metrics  began  in  March  1995.  The 
working  group  completed  development  of  the  performance  indicator  package  in 
June  and  the  prototype  software  program  was  installed  on  USS  GEORGE 
WASHINGTON  for  test  and  evaluation  in  September  1995.  Based  on  Battle 
Group  staff  recommendations,  the  software  program  was  continuously  refined 
during  the  training  workup  period  with  the  “sail  away”  version  for  deployment 
installed  in  January  1996.  Formal  data  collection  for  GPRA  purposes 
commenced  on  departure  for  deployment  on  27  January  1996. 
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III.  Performance  Plan  Development  and  Execution 


Chief  of  Naval  Operations  (CNO)  Instruction  3501.316  of  February  1995 
provided  the  working  group  with  the  doctrinal  foundation  for  development  of 
the  performance  measurement  plan.  This  instruction  delineates  Navy  policy  on 
those  mission  critical  tasks  a  Carrier  Battle  Group  must  be  capable  of  executing 
in  both  peacetime  presence  and  contingency  operations.  Given  the  limited  time 
available  to  meet  0MB ’s  14  April  submission  date  and  CINCLANTFLT’s 
objective  of  minimizing  added  workload  on  the  deploying  Battle  Group  staff, 
selective  boundaries  were  imposed  to  judiciously  control  the  scope  of  the 
tasking  while  ensuring  the  objectives  of  the  GPRA  pilot  plan  process  were  fully 
met.  These  subjective  boundaries  provided  a  framework  for  the  working  group 
and  are  described  in  detail  in  Appendix  A. 

The  mechanism  for  developing  the  performance  plan  was  a  CINCLANTFLT 
and  Commander,  GEORGE  WASHINGTON  Battle  Group  co-chaired 
Performance  Indicator  &  Metrics  Fleet  Working  Group.  Standing  working 
group  membership  included  warfare  area  subject  matter  experts  (SMEs)  from 
CINCLANTFLT,  Commander,  SECOND  FLEET,  and  the  GEORGE 
WASHINGTON  Battle  Group,  carrier  airwing,  and  destroyer  squadron  staffs. 
As  specific  mission  critical  task  requirements  dictated  the  need  for  additional 
expertise,  the  standing  working  group  was  augmented  with  selective  SME 
representation  from  assigned  aircraft  squadrons,  ships  and  submarines. 

As  stated,  CNO  Instruction  3501.316  provides  approved  policy  on  the  required 
operational  capabilities  that  a  Carrier  Battle  Group  must  be  able  to  demonstrate 
as  a  forward  element  of  our  National  Military  Strategy.  These  required 
capabilities  are  stated  in  the  form  of  13  Critical  Tasks  listed  below: 

Air  Superiority  Theater  Ballistic  Missile  Defense 

Maritime  Superiority  Special  Operations 

Power  Projection  Amphibious  Operations 

Surveillance  &  Intelligence  hand  Forces  Insertion 

Command  &  Control  Mine  Warfare 

Peacetime  Presence  Combat  Search  &  Rescue  (SAR) 

Sustainment 

Figure  1  illustrates  the  process  model  used  by  the  Fleet  Working  Group  to 
develop  the  Battle  Group  performance  plan  and  its  associated  indicator  & 
metrics  package.  The  critical  tasks  in  bold  above  were  approved  by 
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CINCLANTFLT  N8  and  the  Battle  Group  Commander  for  inclusion  in  the 
performance  plan.  This  was  done  purposefully  to  limit  the  tasking  to  a 
manageable  level  while  ensuring  the  critical  operational  capabilities  of  the 
Battle  Group  were  adequately  addressed.  Those  critical  tasks  in  italics  were  not 
selected  for  inclusion  for  various  reasons.  As  examples,  the  GEORGE 
WASHINGTON  Battle  Group  was  not  configured  to  conduct  the  Theater 
Ballistic  Missile  Defense  mission;  critical  personnel  and  equipment  capabilities 
needed  to  conduct  Combat  SAR  and  Special  Operations  were  addressed  in  other 
selected  critical  tasks;  and  given  that  non-organic  Mine  Warfare  forces  (e.g., 
Mine  Counter  Measure  ships  and  helicopters)  are  not  routinely  attached  to  the 
Battle  Group,  assessing  day  to  day  performance  in  that  critical  task  would  not 
have  been  possible. 


PERFORMANCE  PLAN  MODEL 


CRITICAL  TASKS 
^PROCESSES) 


FIGURE  1 


For  each  selected  critical  task,  the  supporting  subtasks  were  reviewed  by  the 
working  group  for  inclusion  into  the  performance  plan,  again  from  the  criteria  of 
priority  of  importance  in  day  to  day  operations  and  overall  project  plan 
manageability.  Once  a  particular  subtask  was  agreed  upon  for  inclusion, 
discussion  focused  on  the  key  indicators  that  would  best  capture  how  well  the 
Battle  Group  was  performing  that  particular  subtask.  As  an  agreed  upon 
working  group  groundrule,  key  indicators  could  be  inputs,  outputs,  or 
process/subprocess  “outcomes”. 
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It  is  important  to  mention  another  working  group  groundrule  at  this  juncture. 
Given  our  lack  of  experience  in  aggregate  level  performance  measurement  and 
our  objective  of  keeping  the  model  as  simple  as  possible,  the  performance 
indicators  were  not  weighted.  As  experience  and  data  accumulated  during 
deployment,  some  minor  program  modifications  were  made  to  account  for 
“natural  weighting”  in  the  configuration  of  the  embarked  airwing,  e.g.,  four  E- 
2C  airborne  early  warning  aircraft  versus  50  F-14  and  F/A-18  strike/fighter 
aircraft.  Each  type  aircraft  is  essential  to  the  airwing  but  the  loss  or  non¬ 
availability  of  two  E-2Cs  in  comparison  to  two  F-14s  or  F/A-18s  generates  a  far 
greater  impact  on  force  capability. 

The  data  selection  process  was  also  governed  by  several  criteria.  First,  no  new 
data  requirements  were  to  be  levied  on  the  Battle  Group  staff  The  data  had  to 
already  exist.  Second,  the  data  had  to  be  immediately  available  to  the  staff  on 
the  carrier  in  either  hard  copy  or  electronic  format.  Once  these  criteria  were 
met,  the  working  group  then  developed  a  graduated  matrix  for  each  subtask, 
defining  the  metric  breakpoints  for  each  indicator  at  specified  levels  of 
performance,  i.e..  Battle  Group  B-rating  levels  1-5.  As  an  example,  Figure  2 
illustrates  the  matrix  developed  for  Air  Superiority  Subtask  3  A  -  Detect, 
monitor,  and  maintain  readiness  to  intercept  aerial  contacts  -  with  the 
performance  indicators  arrayed  horizontally  at  the  top  and  the  percentage 
breakpoints  for  B-rating  levels  1-5  aligned  vertically  under  each  indicator. 

AIR  SUPERIORITY 

Sub-task  ‘3a’  Matrix 

SHIPES  AraCRAFT  MISSION  COMraJElWIfS  LINK  % 

3DRADAR  SYSTEM  MCRATES  RATES  TRAP  EFFEC-  AIR 


B^Rating 

CASREPs 

CASREPs 

VAW 

VFA^A 

VAQA^Q 

VAW 

VFA^FA 

VAQA^Q 

Aq 

TIVENESS 

UNK 

1 

0% 

0% 

90% 

90% 

90% 

96% 

96% 

96% 

95% 

95% 

0% 

2 

10% 

10% 

83% 

83% 

84% 

92% 

92% 

92% 

90% 

90% 

10% 

3 

20% 

20% 

75% 

75% 

77% 

88% 

88% 

88% 

85% 

85% 

20% 

4 

30% 

30% 

70% 

70% 

72% 

84% 

84% 

84% 

80% 

80% 

30% 

5 

>30% 

>30% 

<70% 

<70% 

<72% 

<84% 

<84% 

<84% 

<80% 

<80% 

>30% 

FIGURE  2 

Through  close  involvement  in  the  developmental  process,  the  Battle  Group 
Commander  provided  sufficient  philosophical  guidance  to  ensure  his 
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performance  goals  would  be  in  conceptual  alignment  with  the  working  group’s 
effort. 

Development  and  formal  approval  of  the  indicator/metrics  package  took 
approximately  four  months.  This  included  development  of  the  graduated  B- 
rating  architecture  containing  the  Fleet  Working  Group  developed  performance 
indicators  and  breakpoints.  Appendix  B  provides  amplifying  information  on  the 
Battle  Group  B-rating  structure. 

Given  time  and  workload  constraints,  contractor  assistance  was  used  by  the  N8 
Directorate  to  facilitate  development  and  execution  of  the  pilot  project. 
Assistance  was  secured  through  an  inplace  contract  vehicle  and  ensured  the 
dedicated  availability  of  Battle  Group  experienced  analysts  to  meet  project 
requirements.  Contractor  support  was  used  to  facilitate  formulation  of  the 
performance  plan;  to  develop  the  Battle  Group  Reporting  System  software 
program  as  the  data  collection  vehicle;  and  to  monitor  day  to  day  data  collection 
efforts  by  the  Battle  Group  staff. 
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IV.  Performance  Goals 

As  required  by  GPRA,  the  Battle  Group  Commander  set  his  performance  goals 
prior  to  the  Battle  Group’s  departure  for  overseas  deployment.  His  stated  goals 
were  to  maintain  the  GEORGE  WASHINGTON  Battle  Group’s  overall 
performance  at  the  B-2  level  and  at  B-2  in  each  of  the  seven  selected  critical 
tasks  for  the  duration  of  the  deployment. 

B-2  was  selected  as  the  benchmark  for  the  Battle  Group  based  on  several 
factors.  First,  historical  experience  in  Battle  Group  operations  indicated  that 
maintaining  the  higher  B-1  performance  level  in  all  seven  critical  tasks  over  a 
six  month  period  would  be  impractical  given  both  cost  and  logistics 
supportability  factors.  Second,  selecting  B-2  as  the  “critical  task”  performance 
goal  conceptually  aligned  our  prototype,  aggregate  level  performance 
assessment  system  to  the  unit  level  “mission”  (M)  readiness  reporting  norm  of 
M-2,  typically  expected  of  a  unit  during  deployed  operations. 

For  individual  ships,  submarines,  and  aircraft  squadrons,  unit  level  readiness 
reporting  is  based  on  the  Joint  Chiefs  of  Staff  s  Status  Of  Resource  &  Training 
Systems  (SORTS).  Aside  from  our  prototype  system,  no  corollary  system  to 
SORTS  is  known  to  exist  within  DoD  for  aggregate  force  level  reporting,  e.g.,  a 
Carrier  Battle  Group. 
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V.  Data  Collection 


Development,  testing  and  installation  of  an  EXCEL  based  software  program  to 
collect  and  manipulate  performance  indicator  data  took  approximately  four 
additional  months.  Formal,  daily  data  collection  commenced  with  GEORGE 
WASHINGTON  Battle  Group’s  deployment  to  the  European  and  Central 
Command  theaters  of  operation  on  27  January  1996  and  continued  until  the 
group  returned  home  to  the  operational  control  of  Commander,  SECOND 
FLEET  on  1 1  July  1996. 

Battle  Group  performance  data  was  collected  daily  as  a  routine  process  by 
designated  Battle  Group  staff  members  throughout  the  deployment.  Given  the 
prototype  nature  of  the  software  program,  all  data  was  inputted  manually  from  a 
stand  alone  computer.  Time  was  not  available  to  develop  and  test  a  software 
program  that  would  meet  established  criteria  for  hosting  on  the  carrier’s  local 
area  network  (LAN).  Although  somewhat  labor  intensive,  data  entry  for  all 
indicators  was  completed  in  about  one  hour  on  average.  Administrative 
coordination  and  data  validation  efforts  required  additional  time. 

The  scope  of  the  data  collection  process  entailed  the  daily  measurement  of  280 
performance  indicators  covering  operational  event  scheduling  results,  weapons 
inventory  versus  allowance,  subprocess  performance  outcomes,  stock  &  liquid 
levels  status,  critical  systems  casualties,  aircraft  mission  capable  rates,  and 
critical  personnel  availability  versus  allowance.  These  categories  of  indicators 
generated  approximately  1,050  discrete  daily  data  elements  that  were 
manipulated  by  the  software  program  to  generate  the  quantitative  score  in  each 
subtask  for  rollup  to  the  critical  task  level. 

Figure  3  on  the  following  page  depicts  the  data  collection  process  utilized  in 
this  pilot.  The  software  system  provided  a  time  stamped  audit  of  daily 
GEORGE  WASHINGTON  Battle  Group  (GWBG  on  Figure  3)  performance  in 
each  critical  task  throughout  the  deployment.  Raw  input  data,  computed  data, 
and  subjectively  overridden  or  annotated  data  were  each  separately  archived  in 
the  software  program. 

Battle  Group  performance  data  was  transmitted  daily  to  CINCLANTFLT  N8 
through  established  satellite  connectivity  paths.  Daily  phone  communication 
was  maintained  with  the  Battle  Group  staff  project  coordinator  to  ensure  any 
software 
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deficiencies  were  rapidly  corrected.  Daily  monitoring  of  the  data  collection 
effort  by  the  N8  staff  ensured  the  quality  of  the  data  was  consistent  throughout 
the  deployment. 


DATA  COLLECTION  PROCESS 


VI.  Results 


Figure  4  indicates  that  the  stated  performance  goals  of  B-2  overall  and  B-2  in 
each  of  the  seven  critical  tasks  for  the  duration  of  the  deployment  were  met  or 
exceeded.  Appendix  B  provides  detailed  information  on  the  qualitative  and 
quantitative  definitions  of  the  B-rating  architecture. 

PERFORMANCE  GOALS  &  RESULTS 


FIGURE  4 

Given  the  dynamic  nature  of  day  to  day  Battle  Group  operations,  periodic 
performance  deviations  below  the  B-2  level  were  experienced  during  the 
deployment.  Deviations  from  the  desired  norm  are  not  uncommon  and  are  often 
beyond  the  Battle  Group  Commander’s  control,  e.g.,  weather  problems,  Rules  of 
Engagement  changes,  equipment  casualties,  etc..  Deviations  from  desired 
performance  levels  are  closely  monitored  on  a  daily /hourly  basis  by  the 
Commander  and  his  staff  for  earliest  correction.  No  significant  performance 
degradations  over  a  sustained  period  were  encountered  by  the  GEORGE 
WASHINGTON  Battle  Group  that  necessitate  an  explanation  for  “non¬ 
achievement”  of  stated  goals.  No  “Command  Overrides”  of  data  computation 
results  were  exercised  by  the  Battle  Group  Commander  during  the  deployment. 

It  is  necessary  to  reemphasize  that  this  pilot  project  was  our  initial  attempt  to 
quantify  a  Battle  Group’s  performance  on  a  scale  of  this  magnitude,  i.e.,  number 
and  breadth  of  performance  indicators.  Prior  year  quantitative  data  of  similar 
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scope  and  quality  was  not  available  for  a  comparative  assessment  of  the 
performance  of  the  GEORGE  WASHINGTON  Battle  Group  with  previous 
Battle  Groups  on  similar  deployments.  Data  collection  during  past  Battle  Group 
deployments  has  typically  been  limited  to  a  small  set  of  indicators  focused 
heavily  on  airwing  operations. 

Appendix  C  contains  sample  displays  provided  by  the  Battle  Group  Reporting 
System.  Included  as  examples  are  the  executive  level  display  of  the  seven 
critical  tasks;  a  sample  pull  down  display  of  the  three  Air  Superiority  Subtasks; 
Subtask  B  (Establish  and  Enforce  No-Fly  Zones)  performance  indicators  & 
quantitative  results;  a  sample  trend  display  for  Subtask  B;  an  extract  of  the  data 
archive  file  for  Subtask  B  indicators;  and  Reason  Code  and  Command  Override 
input  displays. 
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VII.  Conclusions  and  Recommendations 


The  following  conclusions  and  recommendations  reflect  the  collective 
experiences  and  lessons  learned  by  both  the  CINCLANTFLT  N8  and  GEORGE 
WASHINGTON  Battle  Group  staffs  over  the  duration  of  this  pilot  project. 

a.  Before  committing  any  resources  to  an  endeavor  of  this  type  and 
magnitude,  the  organization’s  leadership  must  make  the  commitment  to  add 
wisdom  from  the  top  to  ensure  the  resultant  process  only  collects/measures  that 
which  truly  matters  to  the  organization’s  mission. 

b.  Insufficient  time  and  resources  were  available  to  fully  prepare  for  a 
prototype  effort  of  this  magnitude.  The  development  of  performance  indicators 
and  metric  standards  requires  dedicated  senior  management  &  SME 
involvement  and  resources.  This  pilot  required  two  unprogrammed,  full  time 
exempt  (FTE)  staff  ceiling  points  at  CINCLANTFLT  in  addition  to  two 
dedicated  contract  support  personnel.  Development  of  a  data  collection  vehicle 
tailored  to  the  organization  can  be  a  major  resource  driver.  Implementation  of 
GPRA  “out  of  hide”  may  well  add  to  staff  workload  and  operating  costs  and 
thus  increase  the  risk  of  non-attainment  of  its  intended  goals. 

-  Recommend  for  future  pilot  programs  that  a  minimum  of  a  full 
year  of  preparation  time  be  allocated  to  ensure  resources  will  be  available  and 
inplace.  Ceiling  points  and  funding  should  be  provided  by  the  sponsoring 
federal  agency. 

c.  Training  in  the  philosophy  and  mechanics  of  developing  a  performance 
plan  should  have  been  provided  to  participating  activities. 

-  Recommend  that  the  Office  of  Personnel  Management’s  GPRA 
training  syllabus  be  provided  to  the  government  activity  embarking  on 
performance  plan  development. 

d.  Data  collection  must  not  be  an  end  unto  itself.  Attempts  to  “measure 
the  universe”  must  be  scrupulously  avoided.  The  manager  should  be  provided 
with  only  that  data  that  truly  matters  to  the  organization’s  mission.  For 
combatant  forces,  the  data  collection  effort  must  provide  a  value  added 
dimension  to  the  daily  assessment  process  and  not  interfere  with  the  Warrior’s 
reason  for  being. 
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e.  Undue  complexity  should  be  avoided  in  developing  a  model  for 
management  decision  purposes.  The  KISS  (Keep  It  Simple,  Stupid)  principle 
should  guide  the  working  group’s  efforts.  Weighting  of  performance  indicators 
must  be  thoughtfully  considered  to  avoid  biasing  the  model. 

-  Recommend  avoiding  adding  complexity  to  performance  model 
development  until  natural  weighting  relationships  are  observed  as  performance 
data  accumulates. 

f.  The  cultural  change  necessary  for  the  organization  to  willingly 
embrace  the  concept  of  quantitative  performance  assessment  can  be  achieved  if 
the  process  is  judiciously  implemented.  Conversely,  if  a  quantitative 
assessment  system  is  allowed  to  become  a  “report  card”  on  manager/activity 
performance,  then  the  benefits  of  GPRA  will  not  be  realized. 

g.  Performance  of  a  combatant  force  can  be  quantitatively  assessed.  Due 
to  the  unique  dynamics  and  variables  facing  the  combatant  commander, 
measuring  a  combatant  organization  requires  more  latitude  in  implementation 
than  required  in  measuring  a  more  static  organization  operating  under  standard 
business  processes. 

For  this  pilot,  CINCLANTFLT  developed  the  concept  of  “projected  output 
capability”  as  a  proxy  for  the  “outcome  measures”  required  by  GPRA.  This 
was  necessary  since  performance  measurement  under  the  concept  of  “outcome 
measures”  would  have  necessitated  actual  combat  operations. 
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VIII.  Summary 


The  GEORGE  WASHINGTON  Battle  Group  pilot  was  successful  in  providing 
the  Battle  Group  Commander  with  a  prototype  quantitative  performance 
assessment  system  that  reasonably  measured  and  aggregated  selective  Battle 
Group  capabilities  at  a  level  more  appropriate  to  the  Flag  level  decision  maker 
than  the  status  quo,  subjective  assessment  process.  Based  on  the  success  of  this 
pilot  project,  CINCLANTFLT  is  refining  and  institutionalizing  the  process  in 
all  its  Carrier  Battle  Groups  and  Amphibious  Ready  Groups. 

From  the  Fleet  Commander  in  Chief  perspective,  the  lessons  learned  from  this 
pilot  are  providing  our  staff  not  only  with  practical  experience  in  performance 
measurement  but  with  a  knowledge  dividend  on  how  we  need  to  proceed  in 
reinventing  our  cost  accounting  practices  to  forge  the  necessary  linkage  between 
performance  and  resources. 
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APPENDIX  A 


BOUNDARIES  TO  PILOT  PLAN  SCOPE 

Due  to  the  limited  time  and  resources  available  to  meet  GPRA  performance 
plan  submission  requirements,  project  boundaries  were  artificially  set  to  control 
the  scope  of  the  pilot  effort  while  still  meeting  the  fundamental  objectives  of  the 
GPRA  performance  plan  pilot  program.  Boundaries  to  scope  included  the 
following: 

a.  Of  the  13  critical  tasks  defined  in  governing  CNO  Instruction  3501.316 
for  a  Carrier  Battle  Group,  CINCLANTFLT  chose  to  measure  those  seven  that 
were  considered  most  relevant  to  the  specific  capabilities  and  configuration  of 
the  GEORGE  WASHINGTON  Battle  Group  and  most  reflective  of  day  to  day 
operational  requirements.  The  seven  designated  critical  tasks  included:  Air 
Superiority,  Maritime  Superiority,  Power  Projection,  Command  & 

Control,  Surveillance  &  Intelligence,  Sustainment,  and  Peacetime 
Presence. 

b.  No  new  data  collection  requirements  to  support  the  GPRA  pilot  were 
levied  on  the  Battle  Group  staff  given  the  magnitude  of  competing  operational 
requirements  related  to  training  workup  and  deployment. 

c.  The  planned  start  time  for  the  performance  plan  was  set  for  September 
1996  to  conform  with  the  FY  96  GPRA  pilot  window  and  the  start  of 
Intermediate  Phase  training  for  the  GEORGE  WASHINGTON  Battle  Group. 
Given  the  time  needed  for  performance  plan  development,  formulation  and 
approval  of  a  supporting  indicators  &  metrics  package,  and  creation  of  a  data 
collection  software  program,  the  formal  collection  period  commenced  on 
departure  for  deployment  in  January  1996.  Collection  of  valid  performance  data 
was  limited  to  approximately  six  months. 

d.  Unit  level  manpower,  personnel,  and  training  (MPT)  indicators  were 
not  heavily  factored  into  the  performance  plan.  Unit  level  training  and 
Certification  principally  occur  in  the  Basic  Phase  of  the  Navy’s  Inter- 
Deployment  Training  Cycle  (Basic,  Intermediate,  and  Advanced  Phases  of 
training).  For  the  purposes  of  this  pilot,  the  general  assumption  was  that 
individual  units  (i.e.,  a  ship,  submarine,  aircraft  squadron)  were  adequately 
manned,  trained  and  certified  in  unit  level  training  requirements  in  order  to  enter 
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Intermediate  Phase  training  as  part  of  the  Battle  Group,  Unit  level  MPT 
mission  area  ratings  from  the  SORTS  system  were  however  selectively 
incorporated  into  the  performance  indicator  data  base  as  quantifiable  inputs. 

e.  Cost  data  gathering  and  correlation  to  performance  were  not  attempted 
in  this  pilot  effort.  Our  effort  focused  on  developing  and  operationally  testing  a 
prototype,  quantitative  performance  assessment  system  designed  to  measure 
aggregate  level,  Battle  Group  mission  capabilities.  Future  CINCLANTFLT 
initiatives  will  address  the  correlation  of  performance  data  to  resources  based  on 
the  experience  gained  in  this  pilot. 
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APPENDIX  B 


Battle  Group  Reporting  System 

The  performance  indicators  chosen  to  measure  aggregate  level  Battle  Group 
capability  in  the  seven  selected  mission  critical  tasks  were  the  subject  of 
benchmarking  by  the  Fleet  Working  Group.  To  “measure”  graduated  levels  of 
performance,  a  rating  scale  architecture  was  developed  that  intentionally 
mirrored  that  of  SORTS,  the  JCS  unit  level  reporting  system.  This  path  was 
primarily  selected  for  two  reasons  -  First,  the  prototype  Battle  Group  level 
system  was  thought  to  have  a  greater  chance  of  acceptance  by  the  deploying 
Battle  Group  staff  if  conceptually  in  line  with  a  “corporate  culture”  attuned  to 
SORTS;  and  second,  a  general  architectural  alignment  with  SORTS  would 
support  potential  incorporation  of  selected  SORTS  unit  level  readiness  data,  i.e., 
mission  area  M-ratings  and  equipment/system  degradation  C-ratings. 

A  Battle  Group  “B”-rating  architecture  was  constructed  with  the  following 
qualitative  definitions: 

B1  -  Fully  capable  of  doing  an  entire  task  or  subtask 

B2  -  Minor  degradation  to  overall  capability 

B3  -  Major  degradation  to  part  of  a  capability  or  multiple 

minor  degradations 

B4  -  Marginal  capability  to  do  an  entire  task  or  subtask 

B5  -  Unable  to  do  a  task  or  subtask 

Through  the  SME  working  group  process,  each  selected  indicator  was 
benchmarked  to  expected,  quantifiable  levels  of  performance  for  each  B-rating 
level.  Established  standards  of  performance  or  policy  for  a  given  “task”  or 
function  where  available  were  incorporated  and  interpolated  breakpoints  for 
each  B-rating  performance  level  developed.  If  a  standard  was  not  available, 
historical  data  was  researched  or  subject  matter  expertise  applied  to  develop  the 
standard  and  its  graduated  performance  breakpoints. 
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A  uniform  B -rating  “range  of  computed  values”  scale  was  then  constructed  to 
quantitatively  define  the  above  narrative  B-ratings  as  follows: 

B1  -  00.0  -  1.00 

B2  -  1.01  -  2.00 

B3-  2.01-3.00 

B4  -  3.01  -  4.00 

B5  -  4.01  -  5.00 

Through  daily  data  input  and  program  computation,  performance  indicator 
results  were  calculated  as  “Today”  values  as  shown  on  the  sample  task/subtask 
displays  in  Appendix  C.  As  quantitative  data  for  each  indicator  accumulated 
during  deployment,  a  performance  “Average”  was  calculated  and  displayed  as 
the  historical  average  to  allow  comparative  assessment  to  “Today’s” 
performance. 

To  qualify  a  performance  degradation  and  archive  it  for  data  audit  and  retrieval 
purposes,  a  “Reason  Code”  structure  with  a  free  text  feature  was  developed  for 
use  by  the  Battle  Group  staff.  This  “Archive  By  Degradation  Cause”  capability 
provided  the  Commander  with  a  running  audit  of  performance  deficiencies  for 
the  deployment. 

Recognizing  that  performance  indicators  are  in  fact  only  “indicators”,  a 
subjective  “Command  Override”  capability  was  also  built  into  the  program  to 
allow  the  Commander  to  “qualify  the  data”  with  his  expert  judgment  if  so 
desired.  Program  software  routines  allowed  incorporation  of  this  feature 
without  disturbing  the  purity  of  the  raw  or  calculated  data  in  question. 

Sample  displays  of  both  the  “Reason  Code”  and  “Command  Override”  features 
are  included  in  Appendix  C. 
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APPENDIX  C 


SAMPLE  PROGRAM  DISPLAYS 

This  appendix  provides  a  series  of  sample  “outcome”  performance  and 
management  displays  provided  by  the  Battle  Group  Reporting  System  to  the 
Battle  Group  Commander.  The  initial  sequence  of  figures  demonstrates  the 
multi-level  aspect  of  the  reporting  system  as  it  cascades  from  the  critical  task 
level  to  subtask,  specific  performance  indicators,  and  individual  data  cell 
levels.  Also  included  are  sample  management  displays  for  performance  trend 
analysis,  deficiency  “Reason  Code”  annotation  and  archiving,  and  “Command 
Override”  deficiency  qualification. 

Figure  C-1  is  the  Executive  Level  summary  display  of  the  seven  selected  critical 
tasks.  From  left  to  right,  it  illustrates  a  color  coded  indicator  of  the  current  B- 
rating  score  (B-1  =  green/green,  B-2  green/yellow,  B-3  yellow/yellow,  B-4 
yellow/red,  &  B-5  red/red);  the  seven  critical  tasks;  the  current  B-rating  “score” 
(e.g.,  B-2);  the  “Today”  roll  up  averaged,  numerical  score  from  the  supporting 
subtask  level;  and  the  historical  “Average”  score.  A  “dot”  appearing  with  a 
current  B-rating  “score”  (e.g..  Maritime  Superiority  B-2)  indicates  a  serious  B-5 
degradation  at  a  lower  subtask  level.  A  pull  down  arrow  at  right  (not  shown) 
allows  access  to  subtask  and  supporting  performance  indicator  level  displays 
which  utilize  the  same  structure. 


BATTLE  GROUP  REPORTING  SYSTEM 
READINESS  STATUS  AS  OF  11  JUL  96 


KJ  Air  Superiority 

B2 

Today 

(1.13) 

Averaae 

(1.29) 

BJ  Maritime  Superiority 

•B2 

(1.72) 

(1.37) 

Hi  Power  Projection 

B1 

(0.96) 

(1.10) 

IE  Peacetime  Presence 

B2 

(1.23) 

(1.29) 

B  Sustainment 

•B2 

(1.11) 

(1.23) 

BLI  Command  &  Control 

B2 

(1.15) 

(1.14) 

Hi  Surveillance  &  Intel 

B1 

(0.95) 

(1.08) 

FIGURE  c-1 

C-1 


Figure  C-2  is  the  pull  down  display  for  the  Air  Superiority  Critical  Task.  It 
shows  the  three  supporting  subtasks  with  the  numerical  score  for  “Today”  and 
the  historical  “Average”  score  to  date. 
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Figure  C-3  provides  the  pull  down  display  of  performance  indicators  supporting 
Air  Superiority  Subtask  B  -  Establish  &  Enforce  No  Fly  Zones.  This  subtask 
uses  the  averaged  score  from  Subtask  A  (Detect,  monitor,  &  maintain  readiness 
to  intercept  aerial  contacts)  plus  indicators  for  critical  equipment/sy stems 
availability  reported  as  “percent  available”  and  converted  by  the  program  to  a 
B-rating  score. 
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Figure  C-4  illustrates  an  extract  of  the  data  cell  archive  file  for  selected  Subtask 
B  indicators.  Raw  data  is  entered  daily  by  designated  staff  members  and 
archived  into  the  data  base  for  averaging  and  comparative  analysis  purposes. 
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Figure  C-5  provides  a  sample  of  the  trending  display  capability  of  the  Battle 
Group  Reporting  System  for  Subtask  B.  The  vertical  axis  displays  the  score 
range  with  the  blue  line  showing  the  average  score  for  the  indicator  over  time  as 
displayed  on  the  horizontal  axis. 
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Figure  C-6  is  the  “Reason  Code”  display  used  by  the  staff  action  officers  to 
document  performance  deficiencies.  Six  reason  code  “bins”  are  provided  to 
categorize  degradations  by  type  cause.  A  free  text  capability  allows  the  action 
officer  to  explain  the  degradation,  operational  impact,  and  expected  time  of 
correction. 
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Figure  C-7  is  the  final  display  and  is  designed  to  allow  the  Battle  Group 
Commander  or  his  principal  warfare  commanders  to  “override”  the  current 
numerical  score  for  an  indicator  based  on  expert  judgment.  Exercising  this 
option  changes  the  B -rating  color  code  display  but  does  not  change  the 
computed  score  for  the  indicator.  A  “flag”  is  provided  on  the  display  to  indicate 
the  override  function  has  been  exercised. 
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